11.2 Markov Chains
125
and ensemble averages is called ergodicity. Note, however, that many biological
systems appear to be frozen in small regions of state space, as a glass, and hence are
nonergodic (cf. Sect. 7.4.2).
Notice some of the difficulties inherent in the above description. For example, we
referred to “an unchanging underlying mechanism”, yet at the same time asserted
that a random process is one which does not depend in a completely definite way
on the independent variable. Yet, who would deny that the coin, whose tossing
generates that most archetypical of random sequences, does not follow Newton’s
laws of motion? This apparent paradox can be shown to be a consequence of dynamic
chaos (Sect. 12.3).
If successive values of yy are not correlated at all, that is,
W2(y1t1, y2t2) = W1(y1t1)W1(y2t2),
(11.2)
etc., all information about the process is completely contained inupper W 1W1 and the process
is called a purely random process.
11.2
Markov Chains 5
In the previous section, we considered “purely random” processes in which succes-
sive values of a variable, yy, are not correlated at all. If, however, the next step of a
process depends on its current state, that is,
W2(y1y2t) = W1(y1)P2(y2|y1t) ,
(11.3)
where upper P 2 left parenthesis y 2 vertical bar y 1 t right parenthesisP2(y2|y1t) denotes the conditional probability that yy is in the range
left parenthesis y 2 comma y 2 plus(y2, y2 + dy 2 right parenthesisy2) after having been at y 1y1 at a time tt earlier, we have a Markov chain
(cf. Sect. 6.2).
Definition. A sequence of trials with possible outcomesbold aa (possible states of the sys-
tem), an initial probability distribution bold a Superscript left parenthesis 0 right parenthesisa(0), and (stationary) transition probabilities
defined by a stochastic matrix upper PP is called a Markov chain. 6
The probability distribution for an rr-step process is
a(r) = a(0) Pr .
(11.4)
5 See also Sect. 6.2.
6 In some of the literature, one finds stochastic matrices arranged such that the columns rather than
the rows sum to unity. The arrow in the top left-hand corner serves to indicate which convention is
being used.